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Abstract 

The aim of this paper is to introduce new statistical criterions for estima- 
tion, suitable for inference in models with common continuous support. This 
proposal is in the direct line of a renewed interest for divergence based inference 
tools imbedding the most classical ones, such as maximum likelihood, Chi-square 
or Kullback Leibler. General pseudodistances with decomposable structure are 
considered, they allowing to define minimum pseudodistance estimators, without 
using nonparametric density estimators. A special class of pseudodistances in- 
dexed by a > 0, leading for a 1 to the Kulback Leibler divergence, is presented 
in detail. Corresponding estimation criteria are developed and asymptotic prop- 
erties are studied. The estimation method is then extended to regression models. 
Finally, some examples based on Monte Carlo simulations are discussed. 

1 Introduction 

In parametric estimation, minimum divergence methods, i.e. methods which estimate 
the parameter by minimizing an estimate of some divergence between the assumed 
model density and the true density underlying the data, have been extensively studied 
(see Pardo (2005) and references herein). Generally, in continuous models, the minimum 
divergence methods have the drawback that it is necessary to use some nonparametric 
density estimator. In order to remove this drawback, some proposals have been made 
in literature. Among them, we recall the minimum density power divergence method 
introduced by Basu et al. (1998), and a minimum divergence method based on duality 
arguments, independently proposed by Liese and Vajda (2006) and Broniatowsi and 
Keziou (2009). The results obtained in the present paper follow this line of research. 

Without referring to all properties of the divergence criterions, we mainly quote 
their information processing property, i.e. the complete invariance with respect to the 
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statistically sufficient transformations of the observation space. This property is useful 
but probably not unavoidable in the minimum divergence estimation based on similarity 
between theoretical and empirical distributions. In this paper we admit general pseu- 
dodistances which may not satisfy the information processing property. The definition 
of the pseudodistance, which is at the start of this work, pertains to the willingness to 
define a simple frame including all commonly used statistical criterions, from maximum 
likelihood to the L 2 norm. Such a description is provided in Broniatowski and Vajda 
(2009). In the present paper we define a class of pseudodistances indexed by a > 0, 
leading for a J, to the Kulback Leibler divergence. The peculiar features of these 
pseudodistances recommend it as an appealing competing choice for defining estimation 
criteria. We argue that by defining and studying minimum pseudodistances estima- 
tors for classical parametric models, respectively for regression models. We present such 
tools for inference with a special attention to limit properties and robustness, in a similar 
spirit as in Toma and Broniatowski (2011). 

The outline of the paper is as follows. Section 2 introduces decomposable pseudodis- 
tances and define minimum pseudodistances estimators. Section 3 presents a special 
class of minimum pseudodistances estimators. For these estimators we study invariance 
properties, consistency, asymptotic normality and robustness. The estimation method 
is applied to linear models for which asymptotic and robustness properties are derived. 
These results are presented in Section 4. Finally, in order to illustrate the performance 
of the proposed method in finite samples, we give some examples based on Monte Carlo 
simulations. 



2 Decomposable pseudodistances and estimators 

We will consider inference in continuous parametric families, since this is the interesting 
and complex case with respect to the case of models with finite or countable support. 
Hence V is a parametric model with euclidian parameter space and we assume that 
all the probability measures Pg in V share the same support, which is included in M. d . 
Every Pg has a density pg with respect to the Lebesgue measure. 

We denote by P e m P the class of probability measures induced by samples, namely 
the class of all probability measures 



1 n 

P n :=-J2 6 ^ 



n 
1=1 

where X±, . . . , X n is sampled according to a distribution on (W 1 , B not necessarily 

in V. In addition to the previous notation it is useful to introduce a family of measures 
Vo associated to distributions generating the data when studying robustness properties. 
Often, such a measure is a mixture of some element in V with a Dirac measure at some 
point x in R d . We also define V + := VUV . 
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Definition 1 We say that D : P ® P + (->■ R is a pseudodistance between a probability 
measures P E V = {P e : 9 E Q} and Q E V + if ®(Po,Q) > 0, for all 9 G 6 and 
ge?+ and D(P , P e -) = 0«/ and only if 9 = 9. 

Definition 2 A pseudodistance D on V <S> P + is called decomposable if there exist 
functionals D° : P h-> R ; 2? 1 : P + h-> R and measurable mappings 

„ . tn>d 



H- R, 9 E Q 

such that for all 9 E and Q G P + iae expectations J pgdQ exist and 



2(P e , Q) = D°(P e ) + &(Q) + J Pe dQ. 



(2.1) 



(2.2) 



A known class of pseudodistances is that introduced by Basu et al. (1998) and 
called the class of power divergences. This class corresponds to 



®(Pe,Q) 



i + i 

a 



>q 



a 



+ -q a+1 } dX for a > 0. 



Note that the pseudodistances (12. 3ft are decomposable in the sense (12.21) with 

1 



£°(P e ) = j p% +1 d\, &(Q) = ^J q a+1 d\ and 



- 1 + 



a 



(2.3) 



(2.4) 



In the next section, we introduce a new class of pseudodistances from which a new 
statistical criterion for inference is deduced. 

Definition 3 We say that a functional : Q i-> for Q = V + U V emp defines 
a minimum pseudodistance estimator (briefly, minD- estimator) if the pseudodistance 
2)(Pg, Q) is decomposable on P®P + and the parameters T®(Q) E minimize D {Pe) + 
J pedQ on 0, in symbols 



Tv(Q) = arginf £°(P e ) + 



In particular, for Q = P n E V emp 

0v,n '■= Tv(P n ) = arginf 

6 

Theorem 1 Every min Q- estimator 



?dQ 



for all QEQ. 
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!D O (P0) + -5>(X 

8=1 



(2.5) 



(2.6) 



0©- = arginf 



n 

© (P fl ) + -J>OXi 
i=i 



is Fisher consistent in the sense that 

Tv(P eo ) = 9 , forall9 EQ. 



(2.7) 



(2.8) 



Proof. Consider arbitrary fixed 6q £ ©• Then, by assumptions, ^(P^) is a finite 
constant. Therefore ( 12. 5ft together with the definition of pseudodistance implies 



arg inf 



arg inf 



V°(P e ) + J Pe dPe 
D°(P e ) + D 1 (P eo ) + 



9 dP ft 



aigmf®(P ,P eo ) = 6 o . 



The decomposability of a pseudodistance S)(Pe,Q) leads to the additive structure 
of the empirical version 



f 1 n 

D(P e , P n ) ~ D°(P e ) + / PedP n = D°(P e ) + - ^ w(*< 

i=l 



(2.9) 



in the definition ( 12. 7p of the min ID-estimators, which opens the possibility to apply the 
methods of the asymptotic theory of M-estimators (cf. Hampel et al. (1986), van der 
Vaart and Wellner (1996), van der Vaart (1998) or Mieske and Liese (2008)). 



3 A special class of minimum pseudodistance esti- 
mators 

3.1 Definitions and invariance properties 

For probability measures P G V and Q G V + consider the following family of pseudodis- 
tances of orders a > 0, 

^(P.Q) = (/ P MP) + ^— } ln (/OK?) - I in (/ pW) ■ (3.1) 

The following basic condition which guarantees the finiteness of the pseudodistances 
9i a (P, Q) is assumed. For some positive (3, 

pf>, q p MP e Li(Q) for all P G V, Q G P + , (3.2) 

where Li(Q) := {/ : R d -> R such that / |/|dQ < oo}. 
We then have: 

Theorem 2 Let the condition AS.Ufy hold for some (3 > 0. T/ien for all < a < (3 , 
9\ a (P,Q) defined in is a family of pseudodistances decomposable in the sense 

D\ a (P,Q) = K(P) + K(Q)-l^(J pW) , (3.3) 
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where 



and the limit relation holds 



JHa(P, Q) ->■ *Ho(P, Q) := J IngdQ - ^ InpdQ for a | 0. 



(3.4) 



(3.5) 



Proof. Under ( 13. 2p . the expressions ln( f g Q dQ), ln( f p a dQ) and J InpdQ appearing 
in (13. ip and ( 13 .5p are finite so that the expressions fR a (P,Q) and £Hq(-P, Q) are well 
defined. Recall that, for arbitrary arguments s,t > and fixed parameters a, b > with 
the property l/a + l/6=lit holds 



St < — + - 

a o 



(3.6) 



with equality if and only if s a = t b . Indeed, from the strict concavity of the logarithmic 
function we deduce the inequality 

1 1 / s" t b 

ln(st) = - In s a + - \nt b < In — + - 

a b \a b 

and the stated condition for equality. 
Taking a > and substituting 



t 



1 + a , 
with a = , b = 1 + a 

a 



(Jp-dA) 1/a ' {Jq b d\) 1/b 
in the inequality ( 13. 6p . and integrating both sides by A, we obtain the inequality 



P a qd\ < ( / p 1+a d\ 



a/(l+a) 



q 1+a dX 



1/(1+") 



with equality if and only if p aa = q b A-a.s., i.e. if and only if p = q A-a.s. Since the 
expression (13. ip satisfies for a > the relation 



V\ a (P,Q) = -<\n 

a 



p 1+a dX 



a/(l+a) 



q l+a dX 



i/(i+«)' 



In / p a qdX } , (3.7) 



we see that 9t a (P,Q) is a pseudodistance on the space V <S> V + . The decomposability 
in the sense of H 3 . 3 1) on this space is obvious and the limit relation 

mo{P,Q) = lim ft a (P,Q) 



results as follows: 



lim %>(P,Q) 

c40 



lim — 

aio a + 1 

1 



In 



p a dP + 



mat + 1; 



In 



q a dQ --In 
a 



In 



lim 

alO a + 1 

c4o a J p a d<5 



p a dP - In / q a dP 



1 

a 



In 



-In 



p a dQ 



ln^dQ = «Ro(P,g). 
P 



Similarly as earlier in this paper, we are interested in the estimators obtained by re- 
placing the hypothetical probability measure Pg in the 9^ a -pseudodistances *R a (Pg, Pe ) 
by the empirical measure P n . Consider therefore the family of minimum pseudodistance 
estimators of orders < a < (3 (in symbols, min ^-estimators) defined as 9 n = T a (P n ) 
for T a {Q) E with Q e Q = V + U V cmp defined by 



arginf, In {$ p a e dP e ) - £ln(/jW)] 
arg inffl - / \np e dQ 



T a (Q) = 

The upper formula is equivalent to 



if < a < (3 
if a = 0. 



(3. 



T a (Q) = arg sup 



where 



C a (0) 

a/{l+a) 



dA 



c a (9) = 

Hence, alternatively, we can write 

argsup, [-^ln (f pgdPg) + ± In (I E^iPflM)] 
argsup e ^™ =1 lnp e (Xi) 



(3.9) 



(3.10) 



if < a < /3 
if a = 



or 



0, 



arg sup e C«(0) 1 J EHi Pe 
argsup e i^" =1 lnp e (Xi) 



if < a < (5 
if a = 0. 



(3.11) 



(3.12) 



Note that, for a J, 0, the upper criterion function in (13. lip tends to the lower 
maximum likelihood criterion. Indeed, 



lim 
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by the l'Hospital rule. 

In the following, we give some invariance properties of min ^-estimators. 

If the statistical model ((R d ,B(R d ));V = (Pg : 9 G 0)) is reparametrized by ■& = 
${0), then the new min ^-estimators d n are related to the original 9 n by d n = d(9 n ). If 
the observations x G X are replaced by y = T(x), where T : (R d ,£(IR d )) H- (R d , B(R d )) 
is a measurable statistic with the inverse T _1 , then the densities 

in the transformed model V = (Pg = PgT' 1 : 9 G 0) with respect to a-fmite dominating 
measure A = AT -1 are related to the original densities pg by 

Pe(y)=Pe(T- 1 y)My), (3.13) 

where Jr{y) = dAT _1 /dA is a generalized Jacobian of the statistic T. If A is the 
Lebesque measure and the inverse mapping H = T _1 is differentiable, then Jr{y) is the 
determinant 

d 



My) 



, H(y) 
dy 



The min 9^ a -estimators are in general not equivariant with respect to invertible 
transformations T of observations, unless a = 0. 

Theorem 3 The min *R a - estimators 9 n in the above considered transformed model co- 
incide with the original min *R a - estimators 9 n , if the Jacobian Jt of transformation is 
a nonzero constant on the transformed observation space. Thus, the min £H a - estimators 
are equivariant under linear statistics Tx = ax + b. 

Proof. For a = the min ^-estimator coincides with the maximum likelihood 
estimator, whose equivariance is well known. For a > 0, by ( 13.1 3p and f)3.12p . 

1 - 

9 n = argsupC^r 1 - V^(T^) 

1 n 

= argsupCJ^)- 1 - Y^p^Xi) J%{TXi). 



n 
1=1 



Comparing with f)3.12p it follows that 9 n = 9 n if y \-> Jr(y) is a nonzero constant. If 
a = 0, then the estimator coincides with the maximum likelihood estimator and its 
equivariance is well known. 



7 



3.2 Limit properties of min ^-estimators 

Define 

R a (do) := sup J h(x,d)dPe (x) 

where 

h( x 9) ■= 

By the Fisher consistency of the functional T a defined in (I3.9p . it holds 



argsup / h(x,9)dPg (x) = 9 
e J 



and 9q is the only optimizer in the above expression, since 9\ a (Pe, Pe ) = implies 
9 = O . 

Define the estimate of R a (9o) through 



f 1 " 

R a (9o) '■= sup / h(x, 9)dP n = sup — > h(X, 
e J en r—f 



where the 9 indicates that the sampling is i.i.d. under P 9o . The min ^-estimator 9 n is 
then defined through 



~ 1 
9 n = arg sup - /i(X; 



en 
i=i 

This optimum need not be uniquely defined. 

The usual regularity properties of the model will be assumed throughout the rest 
of the paper, namely: (i) The density pe(x) has continuous partial derivatives with 
respect to 9 up to 3th order (for all x A-a.e.). (ii) There exists a neighborhood N(9 ) 
of 9 such that the first, the second and the third order partial derivatives (w.r.t. 9) 
of h(x, 9) are dominated on N(9 ) by some P^-integrable functions, (iii) The integrals 
J(d 2 /d9 2 )h(x,9 )dPe Q (x) and J(d/d9)h(x,9 )(d/d9yh(x,9 )dP eo (x) exist. 

Theorem 4 Assume that the above conditions hold. 

(a) Let B := {9 E 0; \\9 — 0q\\ < n -1 / 3 }. Then, as n — » oo ; wift probability one, the 
function 9 t— y - ^™ =1 h(Xi, 9) attains a local maximal value at some point 9 n in 
the interior of B, which implies that the estimate 9 n is n 1 ^ 3 -consistent. 



(b) ^Jn \ 9 n — 9qJ converges in distribution to a centered multivariate normal random 
variable with covariance matrix 

V = S~ 1 MS- 1 (3.14) 
withS := - J\d 2 /d9 2 )h(x,9 )dP eo {x) and M := J{d/d9)h{x,9 ){d/d9) t h{x,9 )dP eo {x). 
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(c) sfn \^R a (9o) — R a (9Q)j converges in distribution to a centered normal variable with 
variance o~ 2 {9 ) = J h(x,6 ) 2 dPo Q (x) — (J h{x, 9 )dPe Q {x)Y ■ 

Proof, (a) A simple calculus give 

J (d/d9)h(x, 9 )dPg (x) = (3.15) 

and 

r (d 2 /d9 2 )h(x,9 )dP 9o (x) = -S. (3.16) 



Observe that the matrix 5* is symmetric and positive definite. 

Let U n := - Y^i=i{® / 9 ), and use (13.151) in connection with the central 
limit theorem to see that 

y/nU n ->• jV(0,M). (3.17) 

Also, let V n := -^^2™ = i(d 2 /d9 2 )h(Xi,9 ), and use (13.161) in connection with the law of 
large numbers to conclude that 

V n -> -S (a.s). (3.18) 

Now, for any 9 = 9o+un~ 1 ^ 3 with \u\ < 1, consider a Taylor expansion of ~ Y^=i h{Xi, 
in 9 around 9 , and use the hypothesis to see that 

n n 

h ( Xi > °^ = n2/3utU n + 2~ 1 n 1 / 3 u*KM + 0(1) {a.s.) 

uniformly on u with \u\ < 1. Now, use (I3TT8|) and the fact that U n = O (V 1 / 2 (log log n) 1 / 2 ) 
(a.s) to conclude that 

n n 

^h{X h 9) -J2 h ( x i,8o) = O (n 1/6 (loglogn) 1/2 ) -2- 1 u t Sun 1/3 + 0(1) {a.s.) 

i=l i=l 

uniformly on u with \u\ < 1. Hence, uniformly on the surface of the ball B (i.e., 
uniformly on u with \u\ = 1), we have 

nP n h{9) - nP n h{9 ) < O (n 1/6 (loglogn) 1/2 ) - 2- 1 cn 1/3 + 0(1) {a.s.) (3.19) 

where c is the smallest eigenvalue of the matrix 5*. Note that c is positive since 5* 
is positive definite (it is symmetric, positive and non singular). In view of (13. 19 j) . by 
the continuity of 9 (-> J2i=i h{X i: 9) — Y17=i ^0^i> ^o) an d since it takes value zero on 
9 = 9q and is asymptotically nonpositive, it holds that asn-> oo, with probability one, 
9 i — y — Ym=\ 9) attains its maximum value at some point 9 n in the interior of the 

ball B, and therefore the estimate 9 n satisfies 9 n — 9( 
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(b) Using the fact that -Eh(^)K^)^) = an d a Taylor expansion of 
n Y^!i=i(P / 9) in 6 n around 6* , we obtain 

1 n 1 n n 

= -Y d {d/d9)h{X i X) = -^(9/^)/ i (X i ^ o )+(^-0 o ) t ^(9 2 /^ 2 )/ i (X,^o)+o p (n- 1 / 2 ). 



n — ' n 

i=l i=l i=l 



Hence, 



(On - 6 ) = -V^VnUn + „(1). (3.20) 
Using (13.17P and A3. 18[) and Slutsky theorem, we conclude then 



where V = S^MS' 1 . 

1 V^n 



(c) A Taylor expansion of R a (6 ) = - Y^=i h{Xh ®) m @n around 6* , using the fact 
that f(d/d6)h(x,e )dP eo (x) = 0, gives 



1 - 

n < s 



n 

i=l 



Hence, 



" i n r 

(R a (0 O ) - P*(#o)) = - k ^ 9 ^ ~ / ^)dPe (x) 

71 i=l 



+ Op(l), 



which by the central limit theorem, converges in distribution to a centered normal 
variable with variance cr 2 (#o) = / h{x, 6o) 2 dPg (x) — (J h(x,9o)dPg (x)) . 



3.3 Robustness results 

We recall that a map T defined on a set of probability measures and the parameter space 
valued is a statistical functional corresponding to an estimator 6 n of the parameter 9 , 
whenever T(P n ) = 6 n . The influence function of the functional T in P measures the 
effect on T of adding a small mass at x and is defined as 

IF(x; T, P) = lim T ( P ^)~ T ( P ) ; 

e^O e 

where P £X = (1 — e)P + e5 x . When the influence function is bounded, the corresponding 
estimator is called B-robust. The gross error sensitivity of T at P is defined by 

7*(T,P):=sup||IF(x;T,P)||, (3.22) 
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the supremum being taken over all x where W(x\ T, P) exists. 

The statistical functional corresponding to the min ^-estimator is 

T a (Q) : =argsupi^^, (3.23) 

where C a {6) := (J pg +1 dX)°^ r[ . Derivation w.r.t. 9 shows that T a (Q) is a solution of 
the equation 

/ \Pe' l Pe - c a (e)p%)dQ = 0, (3.24) 

where c a (9) := { P i+i dX and pe is the derivative with respect to 9 oipe. 
j p g d\ 

Theorem 5 The influence function of T a in Pq is given by 

lF(x;T a ,P e ) = M a {6)- 1 \p a f\x)p e {x) - c a {9)p a e {x% (3.25) 

where 

MM = jptW - J^^y . (3.26) 
J J p e + dX 

Proof. Consider the contaminated model P ex = (1 — e)Pg — e5 x . Then 

(1 - e) J {vl:l~ P J Ta{hx) ~ ^ T ^))pl (P JdP e + 

+£ K~Jp £ J x)p T ai p s J x ) - c a (T a (P £X ))p« a(Ps Jx)} = 0. 

Derivating with respect to e in the above display and taking the derivative in e = 0, 
after some calculations, we obtain 

IF(x; T a ,P e ) = M a {6)- l \pr\x)p e {x) - c a {9)p a e {x)\, (3.27) 

where M a (6) is given by the formula f)3.26p . 

Beside the influence function, the breakdown point provides information about the 
robustness of an estimator. The breakdown point of an estimator 6 n of a parameter 6q is 
the largest amount of contamination that the data may contain, such that 6 n still gives 
some information about 9q. Following Maronna et al. (2006) (p. 58), the asymptotic 
contamination breakdown point of an estimator 9 n at Pe , denoted by e*(9 n ,9o), is the 
largest e* G (0, 1) such that for e < e* , T((l — s)Pq + eP) as function of P remains 
bounded and also bounded away from the boundary <96 of 9. Here, T((l — s)P$ + eP) 
is the asymptotic value of the estimator at (1 — e)Pe + sP by means of the convergence 
in probability. The definition means that there exists a bounded and closed set K C 
such that K n <90 = and 

T((l - e)P eo + eP) E K, for all e < e* and all P. 
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Figure 1: Influence functions and gross error sensitivity of min ^-estimators of the 
scale parameter a = 1 from the normal model, when m = is known 



3.3.1 Scale models 



(a) Standard deviation of univariate normal. Consider the scale normal model with 
known mean m and take 9 = a. The influence function (I3.25|) takes on the form 



lF{x;T a ,P a ) 



cr a 



+ If I 2 



x — m 



a 



a + 1 



a l x — m 



exp | -- 



a 



The gross error sensitivity of T a in P a is given by 

a(a + l) 3 / 2 a(a + lf/ 2 



7*(T a ,P (T ) = max 



a 



exp 



3a + 2 
"2(a + T) 



(3.28) 



(3.29) 



independently upon the value of m. 



Figure [T] presents influence functions IF(x; T a , P a ), in the case of the scale normal 
model with the known mean m — 0, when a — 1. For the same model, the gross error 
sensitivity of the min JHc-estimator, as function of a, is represented. The gross error 
sensitivity attains its minimum value j*(T a ,P a ) = 1.600413, for a = 0.81648. This 
means that the min ^-estimator corresponding to a = 0.81648 is the most B-robust 
estimator within the class of the min ^-estimators of a — 1. 
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Figure 2: Influence functions and gross error sensitivity of min ^-estimators of the 
scale parameter 9 = 1 from the exponential model 



On the other hand, the asymptotic relative efficiency of T a is 

ARE(T Q , P a ) = ^f^Vi? ZoT - ( 3 ' 3 °) 
(a + l) 6 {3a z + 4a + 2) 

Results for different values of a are given in the first row of Table 1. Note that, when 
a increases, the asymptotic relative efficiency of the estimator decreases. Therefore, 
positive values of a close to zero will ensure high efficiency and in the meantime the 
B-robustness of the estimator. For example, the asymptotic relative efficiency of the 
estimator is 0.975 for a = 0.1, respectively 0.919 for a = 0.2. For both cases, the 
min ^-estimator is B-robust. 

(b) Exponential model. Consider the exponential model with density pe(x) = 
|exp(— |), x > 0. The influence function of a min Dt^-estimator of the parameter 9 
is 

IF(x; T a , Pe) = 6 {a + l) 3 (| - -^j exp (-^) (3.31) 

and the corresponding gross error sensitivity is 

*(r p\ 1 ) 3 ( 2o + l\ 
7 [T a , P e ) = 9 exp — . (3.32) 

a \ a + 1 J 

In Figure [2J for different values of a, influence functions of min D^-estimators of the 
parameter 9 = 1 from the exponential model are represented. The gross error sensitivity 
(13.32p as function of a is also represented. The most B-robust estimator over the class 
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Figure 3: Influence functions and gross error sensitivity of min ^-estimators of the 
location parameter m = from the normal model, when a — 1 is known 



of min 9l a -estimators of 6 = 1 is associated to a = 0.707, case in which the gross error 
sensitivity takes on the value 1.710. 

The asymptotic relative efficiency of T a in P e is given by 



As in the case of the scale normal model, the efficiency remains high for small a. Thus, 
positive values of a close to zero will ensure high efficiency and the B-robustness of the 
estimation procedure. 

3.3.2 Location models 



(c) Mean of univariate normal. Letting pg be the Af(m, a) density with known a, 
the influence function of a min ^-estimator of the location parameter 9 = m is 



ARE(T Q , Pq) 



(2a + l) 3 



(3.33) 



(a + l) 4 (2a 2 + 2a + l)' 




(3.34) 



and the gross error sensitivity is 



7 *(T C 



P m ) = (a + l) 3 / 2 ^exp(-l/2), 



(3.35) 
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Table 1. 

Asymptotic relative efficiencies of the min Dt^-estimators 



a: 0.00 0.02 0.05 0.10 0.20 0.25 0.5 1.00 

Model 

Normal a 1.00000 0.99884 0.99321 0.97543 0.91922 0.88527 0.70572 0.43301 

Exponential^) 1.00000 0.99846 0.99096 0.96741 0.89412 0.85070 0.63209 0.33750 

Normal m 1.00000 0.99942 0.99660 0.98762 0.95862 0.94060 0.83805 0.64951 

Mean of Af 2 (m,V) 1.00000 0.99923 0.99547 0.98353 0.94521 0.92160 0.79012 0.56250 

Mean of M 3 {m,V) 1.00000 0.99903 0.99434 0.97946 0.93199 0.90297 0.74493 0.48713 

Mean of Af 4 (m, V) 1.00000 0.99884 0.99321 0.97541 0.91896 0.88473 0.70233 0.42187 



independently from the value of m. 

In Figure [31 for a = 1 and different values of a, influence functions of min01 a - 
estimators of the location parameter m = are represented. The gross error sensitivity 
(I3.35P as function of a is also represented. Note that, when a — 1, the most B-robust 



estimator over the class of min ^-estimators of m is associated to a = 0.4999836, 
regardless the value of m. 

The ^-function of a min ^-estimator of the parameter m, given by the formula, 

... . a _ 3a + 2 , _ a ( Oi f X ~ i . . i i 

ip a (x,m) = a (a + l) 2 ^ 1 )^ «+i (V27r) "+ 1 (x - m) exp (3.36) 




2 V o 



is redescending w.r.t. x. Then, the asymptotic breakdown point of the corresponding 
estimator is 0.5 according to the results regarding redescending M-estimators of location 
parameters presented in Marona et al. (2006), p. 59. 
The asymptotic relative efficiency is 

(2a + 1) 3/2 

ARE(T Q , P m ) = \ \ > . (3.37) 

Efficiency calculations are presented in the third row of Table 1. Small a min 9^ esti- 
mation continues to retain high efficiency. 

(d) Mean of multivariate normal. The family is J\f p (m,V). The influence function 
of a min D^-estimator of the mean vector m, when V is known, is 

IF(ar; T a , P m ) = (Va + T) p+2 (x - m) exp (-|(^ - mfV^ix - m)) . (3.38) 

This is a bounded function w.r.t. x, meaning that all min D^-estimators of the mean 
vector m are B-robust. In Figure H] the norm of the influence function (13.381) . when 
m = (0, 0)', V = diag(2, 1) and a = 0.2, is represented. 
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Figure 4: The norm of the influence function of a min 9l Q -estimator of m — (0, 0)*, when 
V = diag(2, 1) 
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The limiting covariance matrix of n 1 / 2 times the min $H Q -estimator of m, when V 
is known, can be shown to be 



a + 1 ^ p+2 



V. (3.39) 



^2a + 1 

Then, the asymptotic relative efficiency of a min ^-estimator of m is 

Thus, one loses efficiency for increasing p if a is kept fixed. In Table 1 efficiencies for 
some values of a and p are presented. Again, small values of a ensure high efficiency 
and B-robustness of the estimations. 



4 min ^-estimators in regression models 

Suppose we have i.i.d. {p + l)-dimensional random vectors (X i; Yj), % = 1, . . . , n, satis- 
fying the linear relation 

Y i = p t X i + U i , (4.1) 

where the U^s are i.i.d. with Af(0, a) and independent of the X^s. Xi and j3 are 
p-dimensional column vectors with coordinates (Xn, . . . ,Xi P ) and (/?i, . . . ,/3 p ), respec- 
tively. Call X the nxp matrix with elements X^ and assume that the distribution of X 
is not concentrated on any subspace, i.e. P(a l X = 0) < 1, for all a^O. This condition 
implies that the probability that X has full rank tends to one when n — > oo and holds 
for example if the distribution of X has density. 

Let P a be the probability measure associated to a random variable Af(0, a) and 
P n (f3) be the empirical measure based on the sample Ui, . . . , U n , where Ui = Yi — f3 t X i , 
i — 1, . . . , n. 

The D\ a pseudodistance between P a and the probability measure Q is 
m a (P a ,Q) = — l —\n (^J fi(x)dP a (x)) + J- + In Q q a (x)dQ(x) 

-Un (^J pZ(x)dQ(x)y (4.2) 
The estimators of the parameters j3 and a are defined by minimizing the following 
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empirical version of the pseudodistance (14. 2ft . 
K n {P ai P n {ft)) = 

1 ln/K +1 ^ + r7r ^Tln/^E^ 



a + 1 



a(a + 1; 



n 



,_, , | dP n (/3)--ln / p£dP„(0) 



a 



— In [ P : +1 d\+ , 1 - In if [-fx-, 

i-ln [ p-+ l d\ + ^—\n(-) ~Y p^U, 

+ 1 y a + l \n J a n ~^ 



- - In 



a 



1 n 

(4.3) 



obtained by replacing in (14. 2p Q with P n (ft) and g(a;) with q(x) = ^ X]j=i bx-Uj- 

Since the middle term in the above display does not depend on ft or a, the estima- 
tors ft and a are defined by 



are inf 
or equivalently as 



a 



^ In | ^ +1 ^A - 1 In i E^(^ - ^ 

\ i=i 



arg sup > 



A simple calculation shows that 



and (14.51) writes as 



av27r) "+ 1 (\ZaT+T) "+ 1 



a (Yi - ft*Xi 



(4.4) 



(4.5) 



(4.6) 



a "+ 1 exp 
p,a . * V 2 \ a 



(4.7) 



Derivating with respect to ft and a, we see that the estimators ft and a are solutions 
of the system: 



x ^ / afYi-fttX, 



i=l 



t Y \ 2 

i 



Yi - ft'X, 



a 



X i = 



E exp 



a fYi-ft'X,, 



i=i 



t Y \ 2 



a 



Yi - &X S 



ty\2 



a 



a + l 



(4.8) 
(4.9) 



Note that, for a = 0, the above system corresponds to the system that define the 
least square estimators of ft and a. 



Figure 5: The function <p{u) corresponding to the regression M-estimators, for different 
values of a. 



The system formed by (14.81) and (14. 9p can be written as 

■n 

5^*(Z i ,0 = 0, (4.10) 

i=l 

where Z± = (Xi, Yj), £ is the (p + l)-dimensional vector with coordinates ((3, a) and 

nZi , ^,(YM) XhX (YM))' ,4,!) 

with 



a 



a 



(u) = exp ^— t^ 2 ^ u an d x( u ) 



u 2 - 



1 



a + 1 



(4.12) 



The redescending nature of the functions <fi and % can be seen in Figure |5] and in 
Figure El 

Let £ = (/3,a). The asymptotic normality of the M-estimator £ can be established 
by using similar conditions with those from Theorem HI Such conditions are satisfied by 
the function 



h{z,0 := a «+i exp [ -- I 



(4.13) 
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Figure 6: The function x( u ) corresponding to the regression M-estimators, for different 
values of a. 

associated to the M-estimator £, reason for which we obtain 

v^(f - ^ AT p+1 (0, S-'MiS-'Y) (AAA) 

where 

M = E^(Z : ^(Z^f and S = E*(Z,£), (4.15) 

4/ being the matrix with entries fyjk '■= 

After some calculations we find that the matrices M and S are 

ct2 V x 



M = \ ^ X „ W+4a+2) ) (4.16) 

(2a+l) 5 /2( Q+ i)2 / 



and respectively 



^V X 




S=-[ (Q+1 n 3 2 ) (4.17) 

(a+l) 5 /2 



where V x = EXX T . 

Thus £ is asymptotically normal distributed with the asymptotic covariance matrix 

(q+l) 3 T/ -l 







- 2 ( (2o+i) r x (a+1 )3 (3a 2 +4a+2) i . (4.18) 



4(2a+l) 5 /2 
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It follows that (3 and a are asymptotically independent and the asymptotic covari- 
ance matrix of f3 is a 2 ^a+iytj? L - 

Denote by T and S the statistical functionals corresponding to the estimators 
and <t, respectively. For a given probability measure P, these functionals are defined 
through the solutions of the system 



J *(z,T(P),S(P))dP = 0, 



where 



y — f3 t x\ ( V — /3 l x 



(4.19) 



(4.20) 



o J \ a 

The influence functions of the functionals T and S are given in the following theo- 



rem: 



Theorem 6 The influence functions of the functionals associated to the min 9l a - estimators 
of (3 and a are 



IF(x ,y ;T,Ps) = a (a + l) 3/2 exp I -- 



a 



a 



lF(x ,y ; S,P ( ) 



[O 



+ If/ 2 



ex P I 



/ at \ 2 



a 



at \ 2 

y - p T x Q 



a 



1 



a + 1 



P^ being the probability measure associated to Z . 



Proof. The system (14. 19j) can be written as 

-T(py 

S(P) 



Ix( y -^) dP(x,v) = 0. 



(4.21) 



We consider the contaminated model P e 
is an arbitrary point from MP x IR. For this model, the system f!4.2ip writes as 



e,x ,y - v 1 - £ ) P ^ + sS (x ,y ), where (x ,y ) 



(1-e) 



y - t(p £ , 



xo,yo. 



X 



S(P E ,x 



yo i 



y - T(P e> 



S{Pe,x , 



xdP^(x, y) + ec, 



dP^(x,y) +ex 



S(P e ,x . 



yo — T(P £:XOt y Yx 



tm > 



S(P e 



x = 



0. 



£,xo,yoJ 
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Derivating with respect to e and taking the derivative in e = 0, after some calculations, 
we find 



a 



= a (a + If' exp [ -— I 



y - I3 l x \ ! 

V x x 



a 



and 

lF(x ,y ;S,Pz) 



[a + lfl 2 fyo-^xo 
7i — X 



(a + If/ 2 ( a fy - P l x 



exp | -- 



y - (3 l x 



a + 1 



Since \ is redescending, the estimator o has the influence function bounded and 
hence is B-robust. On the other hand, IF(xo, yo', T, P^) will tend to infinity only when 

xo tends to infinity and yo ~^ x ° < k, for some k. This means that large outliers have 

no influence on the estimates. 



5 Simulation results 



In this section we present some simulation studies in order to illustrate the performance 
of min ^-estimators in finite samples. 

First, we considered the scale normal model with known mean. We estimated the 
scale parameter a by using the min ^-estimator which is obtained as solution of the 
equation 



E 



Xj — m 



a + 1 



exp -- 



a ( Xj — m s 



0, 



(5.1) 



m being the known mean. 

To make some comparisons, we also considered the minimum density power di- 
vergence estimator of Basu et al. (1998) (in the present paper we will denote it by 
min iDa-estimator). For the scale normal model, this estimator is solution of the equa- 
tion 



a 



+ 1)3/ 2 



(a 



1 

n ^ 



Xa — m 



- 1 



exp 



a 
2 



Xj — rri 



a 



0, (5.2) 



In a first Monte Carlo experiment, 5000 samples of size n = 100 were generated from 
the scale normal model A/"(0, 1) with mean m = known, a = 1 being the parameter 
to be estimated. In a second Monte Carlo experiment we generated 5000 samples with 
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100 observations, for each sample 95 observations being generated from A/"(0, 1) and 
5 from jV(2, 1), and then we generated 5000 samples with 100 observations, for each 
sample 90 observations being generated from jV(0, 1) and 10 from jV(2, 1). For each 
sample we computed min ^-estimators and min2) Q -estimators corresponding to a G 
{0.02, 0.05, 0.1, 0.2, 0.25, 0.5, 1} and the MLE for a = 0. 



In Table 2 we present the mean estimated scale a and simulation based estimates 
of the MSE defined by 



where n s denotes the number of samples (5000 in our study) and Oi represents an 
estimate of a = 1 obtained on the basis of the ith sample. 

As it can be seen, both the min O^-estimators and minS) Q -estimators perform well 
under the model. Under contamination, the min ^-estimator with a = 1 gives the best 
results in terms of robustness, while keeping small empirical MSE. However, the min y\ a - 
estimators, as well as the min ©^-estimators, exhibit outlier resistance properties even 
for small values of a. For example, in the case of 5% contamination, the estimates of 
a = 1 obtained for a = 0.2 are 1.07494, respectively 1.07562, fairly close to the estimates 
obtained for a = 1. In this case, the min ^-estimator combines robustness with the 
asymptotic relative efficiency 0.91922. 

Similar results are presented in Table 3 and Table 4, where 5% or 10% from data 
come from the contaminating distribution A/"(0, 3) or from 8iq. When the contamining 
distribution is 5 W , the min 9^-estimators have strong robustness properties, and this 
can also be explained by the influence function which is redescending, as it can be seen 
in Figured! 

In the second example, our estimation method is applied to the location normal 
model A/"(0, 1), a — 1 being known. Here we compute the min ^-estimates as solutions 
of equation 



We consider the case of no outliers and the cases of 5% or 10% outliers coming from 
the model A/"(2, 1). The results are given in Table 5. Again, the choice a = 0.2 provides 
robustness and high efficiency of the estimation procedure. When the outliers come 
from 5±o, we obtain very good results in terms of robustness, even for very small values 
of a, as it can be seen in Table 6. These results are in accordance with the redescending 
nature of the influence functions represented in Figure El 

Our examples show that increasing a leads to estimators which are far more robust 
than the maximum likelihood estimator. The simulation results suggest that a between 
0.1 and 0.25 provides competitive estimators in terms of robustness and efficiency. 




(5.3) 




(5.4) 
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Table 2. 



Simulation results for min D^-estimators, min 5) Q -estimators and MLE of the 
parameter a — 1 when data are generated from the model jV(0, 1), when 95 data 
are generated from the model A/"(0, 1) and 5 data from jV(2, 1), respectively when 
90 data are generated from the model Af(0, 1) and 10 data from Af(2, 1). 

no outliers 5% outliers 10% outliers 

~§ MSE ~§ MSE ~§ MSE 



MLE 



a 










.99763 


0. 


.00503 


1 


.09289 





.01446 


1. 


.17999 


0. 


.03888 


min 


K a 


























a 




0.02 





.99987 


0. 


.00501 


1 


.09216 





.01420 


1. 


.17886 





.03827 


a 




0.05 


1 


.00022 


0. 


.00504 


1 


.08902 


0. 


.01357 


1. 


.17445 





.03663 


a 




0.1 


1 


.00069 


0. 


.00514 


1 


.08398 





.01272 


1. 


.16712 





.03412 


a 




0.2 


1 


.00122 


0. 


.00545 


1 


.07494 


0. 


.01162 


1, 


.15310 


0. 


02999 


a 




0.25 


1 


.00137 


0. 


00566 


1 


.07100 





.01131 


1. 


14659 


0. 


.02835 


a 




0.5 


1 


.00142 


0. 


.00710 


1 


.05610 


0. 


.01122 


1. 


.11981 


0. 


02348 


a 




1 





.99956 


0. 


.01173 


1 


.03931 


0. 


.01494 


1. 


.08746 


0. 


.02315 


min 


D Q 


























a 




0.02 





.99977 


0. 


00463 


1 


.09233 


0. 


01398 


1. 


.17940 


0. 


.03828 


a 




0.05 


1 


.00012 


0. 


.00467 


1 


.08926 


0. 


01336 


1. 


.17505 


0. 


03665 


a 




0.1 


1 


.00060 


0. 


.00477 


1 


.08434 


0. 


.01252 


1. 


.16784 


0. 


.03415 


a 




0.2 


1 


.00123 


0. 


00508 


1 


.07562 


0. 


.01144 


1. 


15420 


0. 


03007 


a 




0.25 


1 


.00146 


0. 


.00527 


1 


.07194 


0. 


.01113 


1. 


.14802 


0. 


.02848 


a 




0.5 


1 


.00217 


0. 


00648 


1 


.05945 


0. 


01093 


1. 


12494 


0. 


02390 


a 




1 


1 


.00326 


0. 


.00882 


1 


.05181 





,01251 


1. 


.10781 





,02259 
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Table 3. 



Simulation results for min D^-estimators, min 5) Q -estimators and MLE of the 
parameter a — 1 when data are generated from the model jV(0, 1), when 95 data 
are generated from the model A/"(0, 1) and 5 data from jV(0, 3), respectively when 
90 data are generated from the model Af(0, 1) and 10 data from jV(0, 3). 

no outliers 5% outliers 10% outliers 

~§ MSE ~§ MSE ~§ MSE 



MLE 



a 







0. 


.99794 





.00498 


1 


.17726 





.04887 


1 


.33251 





.13507 


min 


K a 


























a 




0.02 





.99749 


0. 


,00493 


1 


.15713 





.03788 


1. 


.30542 





.11284 


a 




0.05 


0. 


.99783 


0, 


00496 


1 


.13024 





.02669 


1 


,26450 





.08485 


a 




0.1 


0. 


.99827 


0, 


,00505 


1 


.09683 


0, 


.01692 


1 


,20663 





.05362 


a 




0.2 


0. 


.99874 


0. 


00536 


1 


.06176 





.01059 


1 


,13522 


0. 


02695 


a 




0.25 


0. 


.99884 


0. 


,00557 


1 


.05226 





.00955 


1. 


.11441 


0, 


,02152 


a 




0.5 


0. 


.99869 


0. 


00709 


1 


.03035 


0, 


00905 


1, 


.06610 


0, 


,01367 


a 




1 





.99670 


0. 


.01198 


1 


.01659 


0, 


.01356 


1. 


.03857 


0, 


,01598 


min 


D Q 


























a 




0.02 


0. 


.99870 


0. 


,00487 


1 


.15614 


0, 


,03690 


1. 


.30385 


0, 


,11189 


a 




0.05 


0. 


.99914 


0. 


00489 


1 


.12922 


0, 


02596 


1. 


.26309 


0, 


,08411 


a 




0.1 


0. 


.99973 


0. 


,00497 


1 


.09665 


0, 


,01677 


1. 


,20657 


0. 


,05379 


a 




0.2 


1 


,00051 


0, 


,00525 


1 


.06368 


0, 


,01083 


1, 


,13856 


0, 


,02810 


a 




0.25 


1 


,00078 


0, 


,00543 


1 


.05509 


0, 


00982 


1, 


.11932 


0, 


,02285 


a 




0.5 


1 


,00143 


0, 


00659 


1 


.03786 


0. 


00906 


1. 


.07956 


0, 


,01526 


a 




1 


1 


,00203 


0, 


,00891 


1 


.03484 





,01101 


1, 


,07089 





,01588 
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Table 4. 



Simulation results for min D^-estimators, min D a -estimators 
and MLE of the parameter a — 1 when the data are generated 
from the model Af(0, 1), when 95 data are generated from the 
model A/"(0, 1) and 5 data from 5iq. 

no outliers 5% outliers 

~§ MSE ~§ MSE 



MLE 



a 







0. 


.99859 





.00497 


2 


.43937 


2 


.07260 


min 


K a 


















a 




0.02 





.99731 


0. 


.00494 


2 


.28640 


1 


.65613 


a 




0.05 


0. 


,99773 


0. 


.00497 


1 


.95068 


0. 


.90877 


a 




0.1 


0. 


.99828 


0. 


.00506 


1 


.03369 


0. 


.01115 


a 




0.2 





.99900 


0. 


.00538 





.99922 


0. 


.00575 


a 




0.25 


0. 


.99923 


0. 


.00559 





.99913 





.00592 


a 




0.5 


0. 


.99968 


0. 


00708 





.99947 


0. 


.00747 


a 




1 


0. 


.99875 





.01176 





.99832 


0. 


.01238 


min D a 


















a 




0.02 


0. 


.99567 


0. 


00506 


2 


.28566 


1. 


.65427 


a 




0.05 


0. 


.99602 


0. 


.00509 


1 


.94971 


0. 


90702 


a 




0.1 


0. 


.99646 


0. 


.00517 


1 


.03533 


0. 


.01155 


a 




0.2 





,99700 


0. 


.00544 


1 


.00358 


0. 


.00585 


a 




0.25 


0. 


,99716 


0. 


00563 


1 


.00522 


0. 


00602 


a 




0.5 





,99755 


0. 


.00678 


1 


.01521 


0. 


.00748 


a 




1 





,99818 


0. 


.00905 


1 


.03344 





.01082 
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Table 5. 



Simulation results for min D^-estimators and MLE of the parameter m = when 
data are generated from the model Af(0, 1), when 95 data are generated from the 
model jV(0, 1) and 5 data from Af(2, 1), respectively when 90 data are generated 
from the model J\f(0, 1) and 10 data from J\f(2, 1). 







no outliers 




5% outliers 




10% outliers 




fh 


MoE 


m 


i\ /ret 1 


m 




MLE 














a = 





0.00158 


0.01004 


0.10161 


0.02054 


0.20116 


0.05063 


min 


K a 














a = 


0.02 


0.00075 


0.01003 


0.09723 


0.01956 


0.19573 


0.04853 


a = 


0.05 


0.00072 


0.01006 


0.09257 


0.01873 


0.18769 


0.04554 


a = 


0.1 


0.00067 


0.01015 


0.08568 


0.01768 


0.17557 


0.04141 


a = 


0.2 


0.00059 


0.01045 


0.07457 


0.01640 


0.15539 


0.03549 


a = 


0.25 


0.00055 


0.01065 


0.07007 


0.01605 


0.14698 


0.03339 


a = 


0.5 


0.00036 


0.01194 


0.05427 


0.01583 


0.11651 


0.02774 


a = 


1 


0.00004 


0.01550 


0.03900 


0.01864 


0.08580 


0.02675 



Table 6. 

Simulation results for min ^-estimators and MLE of the parameter m = when 
data are generated from the model Af(0, 1), when 95 data are generated from the 
model jV(0, 1) and 5 data from 5 W , respectively when 90 data are generated from 
the model A/"(0, 1) and 10 data from 5 W . 







no outliers 




5% outliers 




10% outliers 




m 


MSE 


m 


MSE 


m 


MSE 


MLE 














a = 





0.00227 


0.00999 


0.50186 


0.26125 


1.00183 


1.01247 


min 


K a 














a = 


0.02 


0.00018 


0.01021 


0.20421 


0.05278 


0.44055 


0.20618 


a = 


0.05 


0.00023 


0.01023 


0.04819 


0.01342 


0.10345 


0.02288 


a = 


0.1 


0.00033 


0.01033 


0.00499 


0.01088 


0.00999 


0.01159 


a = 


0.2 


0.00054 


0.01064 


0.00106 


0.01111 


0.00147 


0.01166 


a = 


0.25 


0.00065 


0.01085 


0.00112 


0.01132 


0.00150 


0.01188 


a = 


0.5 


0.00118 


0.01219 


0.00158 


0.01275 


0.00194 


0.01336 


a = 


1 


0.00198 


0.01582 


0.00230 


0.01658 


0.00271 


0.01740 
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